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ABSTRACT 


An examination 


of the Wald stopping bounds for the 



Sequential Probability Ratio Test (SPRT) is made by compar- 
ing results obtained from Monte Carlo simulations of 
sequential sampling tests with results obtained using 
Wald formulations. Operating Characteristic, ASN, and 
V[N] values are presented for tests sampling from each of 
eight Binomial, 14 Exponential, and 24 Normal distribu- 
tions. An extensive bibliography of references associated 
with SPRT is included. 



iY 

POSTGRADUATE SCHOOD 
trY, CAT IP. 93940 



TABLE OP CONTENTS 

I. OBJECTIVE g 

II. HISTORY OP SPRT 10 

III. THEORETICAL BACKGROUND OF SPRT 12 

A. DEFINITION OP SPRT 12 

B. DERIVATION OF STOPPING BOUNDS 1 3 

C. RANDOM WALK OP THE OBSERVATION RESULTS li| 

D. THE OC FUNCTION OP THE SPRT li| 

E. THE EXPECTED SAMPLE SIZE OF THE SPRT 15 

P. THE VARIANCE OF THE SAMPLE SIZE OP 

THE SPRT l6 

IV. DISTRIBUTIONS INVESTIGATED 1 9 

A. BINOMIAL DISTRIBUTION U 

B. NORMAL DISTRIBUTION 21 

C. EXPONENTIAL DISTRIBUTION 22 

V. PROCEDURES 2l) 

A. DETERMINATION OP NUMBER OF REPLICATIONS 24 

B. COMPUTER SIMULATION 26 

VI. CONCLUSIONS AND RESULTS 30 

COMPUTER PROGRAM -- APPROXIMATE BINOMIAL 78 

COMPUTER PROGRAM — APPROXIMATE NORMAL 80 

COMPUTER PROGRAM — APPROXIMATE EXPONENTIAL 8 l 

COMPUTER PROGRAM — EXACT BINOMIAL 82 

COMPUTER PROGRAM — EXACT NORMAL 84 

COMPUTER PROGRAM — EXACT EXPONENTIAL 86 



3 



BIBLIOGRAPHY 



88 



INITIAL DISTRIBUTION LIST 94 

FORM DD 1473 95 



4 



LIST OF TABLES 



HYPOTHESIS ERROR 



i- ; "j r ‘ i Li 1 : . ons 


NULL 


ALT. 


TYPE I 


TYPE II 






‘ p 


Li!v.'i-iIAL 


.15 


.30 


.01 


. 01 





_ 


37 


BINOMIAL 


.15 


.30 


.01 


.10 


— 


- 


33 


BINOMIAL 


.15 


.30 


.01 


.20 


— 


- 


34 


BINOMIAL 


.15 


o 

on 


. 10 


. 01 


— - 


- 


35 


BINOMIAL 


.15 


.30 


. 10 


.10 


— 


- 


36 


BINOMIAL 


.15 


.30 


. 10 


.20 


— 


- 


37 


BINOMIAL 


. 10 


.20 


.10 


.10 


— 


- 


38 


BINOMIAL 


.30 


• 70 


.10 


. 10 


— 


- 


39 


EXPONENTIAL 


1.0 


1.5 


.01 


.01 


— 


- 


40 


EXPONENTIAL 


1.0 


1.5 


.01 


.10 


— 


- 


41 


EXPONENTIAL 


1.0 


1.5 


.10 


.01 


— 


- 


42 


EXPONENTIAL 


1.0 


1.5 


.10 


.10 


— 


- 


43 


EXPONENTIAL 


1.0 


2.0 


.03 


.05 


— 


- 


44 


EXPONENTIAL 


1.0 


2.0 


.05 


.03 


— 


- 


45 


EXPONENTIAL 


1.0 


2.0 


.05 


.05 


— 


- 


46 


EXPONENTIAL 


1.0 


2.0 


. 10 


. 01 


— 


- 


47 


EXPONENTIAL 


1.0 


2.0 


.10 


.10 


— 


- 


48 


EXPONENTIAL 


1.0 


3.0 


.10 


.01 


— 


- 


49 


EXPONENTIAL 


1.0 


3.0 


.10 


. 10 


— 


- 


50 


EXPONENTIAL 


1.0 


5.0 


.03 


.05 


— 


- 


51 


EXPONENTIAL 


1.0 


5.0 


.05 


.03 


— 


- 


52 


EXPONENTIAL 


1.0 


5.0 


.05 


.05 


— 


- 


53 


NORMAL 


0.0 


0.5 


.010 


.010 


— 


- 


54 


NORMAL 


0.0 


0.5 


.010 


.100 


— 


- 


55 


NORMAL 


0.0 


0.5 


.100 


.010 


— 


- 


56 


NORMAL 


0.0 


0.5 


.100 


.100 


— 


- 


57 


NORMAL 


0.0 


1.0 


.001 


.001 


— 


- 


58 


NORMAL 


0.0 


1.0 


.001 


.010 


— 


- 


59 


NORMAL 


0.0 


1.0 


.001 


. 100 


— 


- 


60 


NORMAL 


0.0 


1.0 


.001 


.200 


— 


- 


61 



5 



NORMAL 


0.0 


1.0 


.010 


. 001 


— 


62 


NORMAL 


o 

o 


1.0 


.010 


.010 


— 


63 


NORMAL 


0.0 


1.0 


.010 


.100 


— 


64 


NORMAL 


0.0 


1.0 


.010 


.200 


— 


65 


NORMAL 


0.0 


1.0 


.100 


.001 


— 


66 


NORMAL 


o 

o 


1.0 


.100 


.010 


— 


67 


NORMAL 


0.0 


1.0 


.100 


. 100 


— 


68 


NORMAL 


0.0 


1.0 


.100 


.200 


— 


69 


NORMAL 


0.0 


2.0 


.010 


.010 


— 


70 


NORMAL 


0.0 


2.0 


.010 


.100 


— 


71 


NORMAL 


0.0 


2.0 


.100 


.010 


— 


72 


NORMAL 


o 

o 


2.0 


. 100 


. 100 


— 


73 


NORMAL 


0.0 


3-0 


.010 


.010 


— 


74 


NORMAL 


0.0 


3-0 


.010 


.100 


— 


75 


NORMAL 


0.0 


3.0 


.100 


.010 


---- 


76 


NORMAL 


0.0 


3.0 


.100 


.100 


— 


77 


Note: A variance 


of 1. 


0 was 


used with all 


Normal 







Distributions . 



6 



PREFACE AND ACKNOWLEDGEMENTS 



The terms "approximate" and "exact" used in this thesis 
are associated with the Wald derivation of the SPRT and 
the author’s Monte Carlo simulation model of a SPRT pro- 
cedure, respectively. These terms are used only to dis- 
tinguish between the two sources of results. 

The computer programs were tested at the Naval 
Postgraduate School’s Computer Center in the period from 
March 1969 to January 1970. All references in this thesis 
to times and other computer items are related to this 
IBM-360 computer. 

I wish to express my sincere appreciation to Professor 
Donald R. Barr, to the personnel at the library of the 
Naval Postgraduate School, and to the computer facility 
and its staff. 



7 



I. OBJECTIVE 



In this thesis the accuracy of the Wald approximate 
decision limits of the Sequential Probability Ratio Test 
(SPRT) are investigated. The Wald approximate limits are 
compared with "exact" emperical limits obtained by Monte 
Carlo simulation. An extensive bibliography of references 
associated with the SPRT is included. 
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II. HISTORY OF SPRT 



In statistical theory the size of a sample may or may 
not be fixed prior to observation of certain sample values. 
If, in a test of hypotheses, the sample size is not fixed 
in advance, the decision to terminate sampling may depend 
upon the values of the previous samples. Such a test is 
said to be sequential. 

The first mention of sequential test procedures was 
by H. F. Dodge and H. G. Romig who, in 1929, constructed 
a Double Sampling Plan. Prior to World War II, there are 
not many entries in the literature concerning sequential 
procedures. During World War II the Statistical Research 
Group of Columbia University operated under a contract 
with the Office of Scientific Research and Developement 
and was directed by the Applied Mathematics Panel of the 
National Defense Research Committee. Milton Friedman and 
W. Allen Wallis, members of the research group, recognized 
the great potentialities and far reaching consequences 
that sequential analysis might have; consequently various 
members of the group, and in particular A. Wald, worked out 
what is known as the SPRT. In the early 19^0’ s many of 
these results were classified, however, the Restricted 
classification was removed in 19^5-' 

Abraham Wald, in 19^3, worked out the basic principles 
for the SPRT [Wald, 19^31 • During the next two years Wald 
continued working on the basic principles of the SPRT, 
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including a general consideration of cumulative sums of 
independent random variables which gives the Operating 
Characteristic (OC) curve of any SPRT, and the character- 
istic function of the number of observations required by 
the test [Wald, 19W . 

During this same period of time, independent work on 
sequential inferences was also conducted in England by 
G. A. Barnard [Barnard, 19*16] who derived general results 
similar to those obtained by the Statistical Research 
Group at Columbia. 

Since the unfortunate death of A. Wald in 1950, there 
have been many varied contributions to the literature of 
sequential methods. These contributions tend to deal 
primarily with specific families of distributions. Many 
of the articles are listed in the bibliography. 
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III. THEORETICAL BACKGROUND OF THE SPRT 



The Sequential Probability Ratio Test of a simple Null 
Hypothesis against a simple Alternate Hypothesis differs 
from fixed sample size hypothesis tests in that it is 
conducted in stages, where a stage constitutes evaluation 
of an observation. At each stage one of three alternatives 
is chosen: (1) discontinue sampling, accept the null 
hypothesis; (2) discontinue sampling, reject the null 
hypothesis; (3) draw another observation. The procedure 
continues until one of the alternatives (1) or (2) is 
chosen. Under quite general assumptions, the probability 
of eventual termination of the SPRT is equal to one. 1 

A. DEFINITION OF SPRT 

Let the distribution of the random variable, X, under 
consideration be given by the density or mass function, 
f(x; 6). Let H q be the Null Hypothesis that 0 = 0 Q and 
H^ be the Alternate Hypothesis that 0 = 0^ > 0 . Therefore 
the density or mass function of X will be f(x; 0 Q ) when 
H q is true and f(x; 0^) when H^ is true. Successive 
independent observations on X will be denoted x^, 
i = 1,2, The SPRT is based on the likelihood ratio, 

n f(x. ; 0 n ) 
n n = ^ f(x i; e o ) 



1 A. Wald, Sequential Analysis, Wiley, N.Y., 19*17, P-157- 
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and two positive numbers A and B, A >1 and B < 1. After 
each observation on X, the procedure for choosing one of 
the three alternatives is: 

1) if II n <B, Discontinue Sampling, Accept H q 

2) if Discontinue Sampling, Accept 

3) if B<n n <A, Draw another observation. 

B. DERIVATION OF STOPPING BOUNDS 

The two constants A and B are determined so that the 

test will have (nearly) the prescribed probabilities , 

a and 3 a of making errors, where a is the probability of 

making a Type I error (Rejecting H q when it is true) and 

3 is the probability of making a Type II error (Accepting 

H q when it is false) . Exact values for A and B could, in 

principle, be obtained from the following equations given 

in the first that H is true and in the second that H n is 
o 1 

true : 

a = P[n i >A] + P[II 2 >A, B<n i <A] + 

3 = Ptn^B] + P[n 2 <B, B<n 1 <A] + 

In practice, approximations for A and B developed by 

2 

Wald are usually used, where a and 3 are specified apoiri. 
The Wald approximate stopping bounds are 

A = 

a 



2 

A. Wald, Sequential Analysis , op.cit., pp . ^0-4^1. 
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The boundaries used in the tests performed for this 
thesis were formulated as two diagonal lines with proper 
intercepts. These boundaries are called the acceptance 
number, A , and the rejection number, R n< These numbers 
are obtained by setting the logarithm of TI n , the likeli- 
hood ratio, equal to the logarithms of A and B. The 
resulting test is the Wald SPRT with stopping bounds A 
and B. 

C. RANDOM WALK OF THE OBSERVATION RESULTS 

At each observation the value of the test statistic 
was tabulated. The stepwise values of the test statistics 
can be graphed with the abscissa being the number, n, of 
observations made and the ordinate being the value of the 
test statistic. The boundaries, A n and R , will limit the 
steps of the random walk. The test terminates when either 
boundary is reached or surpassed by a value of the test 
statistic . 

D. THE OC FUNCTION OF THE SPRT 

The Operating Characteristic (OC) Function, L(0), is 
defined as the probability that the sequential test will 
lead to acceptance of H q when 0 is the true value of the 
parameter. Using the approximations on the stopping 

^A . Wald, "Sequential Tests of Statistical Hypothesis, 
Annals of Math. Stat ., Vol. 16, No. 2, 19^5, pp . 160-62. 
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bounds mentioned above, Wald showed that the OC function 



could be approximated by 




( 1 ) 



where h(0) is a non-negative real number such that 



f 



00 



[ f(x; eT) ] f(x > 9)dx = 1 > 



f(x; 0 1 ) h ( 6 ) 



(2) 



— 00 



or the equivalent summation in the case f is a mass function. 

It is sometimes difficult in practice to obtain the 
value for h(0) for each of various given values of 0, in 
such cases a "reverse" process may be used: set h(0) 

equal to a non-zero real number and compute a corresponding 
value of 0. This technique was used in computing L(0) for 
the Binomial and Exponential distributions in this thesis. 

E. THE EXPECTED SAMPLE SIZE OF THE SPRT 



eventually terminates. Thus, using the approximate boun- 
daries, A and B, and disregarding the "excess" of over 
these boundaries at termination. 



As mentioned above, with probability one the SPRT 



P [Ln B = Z | 0] = L( 0 ) 

P[Ln A = Z | 0] = 1 - L(0) 



Wald, Sequential Analysis, op.cit., pp. 48-52, 

161-64. 
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where Z 



and where in turn N is the 



N N 

= E Ln II. = Z Z . , 
i=l 1 i=l 1 

sample size required for termination. Therefore, the 
conditional expected value of Z, given 0, is approximately 

E [Z I 8] = [1-L(6)] Ln A + L(6)Ln B . 

5 

Utilizing Wald's Fundamental Identity, the conditional 
expected value of Z, given 0, can be written 



N 

E [Z | 6] = E[ E Z ± | 6 ] = [E] N E[Z i | 6] 



The Expected Sample Size for the test is therefore given 
approximately by 



E[Z| 0] j, [1-L( 6 ) ] Ln A + L(0)Ln B 
E[Z 1 | 0] E[Z 1 |0] 



If E[Z. | 0] = 0, one may approximate the Expected Sample 
Size by 



E[N] 



= e[z 2 | e] 

E[Z 1 2 | 0] 



6 



P. VARIANCE OP SAMPLE SIZE OP THE SPRT . 



It would appear that the Variance of the Sample Size, 
(N), could be approximated in certain cases using an 



. Wald, Sequential Analysis , op.cit., pp. 159-60. 

^A. Wald, "Differentiation under the Expectation Sign in 
the Fundamental Identity of Sequential Analysis," Annals of 
Math. Stat ., Vol. 17, No. 4, p. <472 , December 1946. 
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approach similar to that for E[N]. During his literature 
review the author encountered only three references to 
such an approximation [Wald, 19^5a] a [Walker, 1950] , 

[Cox and Roseberry, 1966a]. 

Since 

V [N ] = E [N 2 ] - E 2 [N] (4) 

an approximation for V[N] may be developed as follows: 

2 

E [N] can be approximated using Eq . (3). An approximate 
2 

value for the E[N ] may be derived from 

p N p N p N 

E [ Z I 0] = E [ ( Z Z.) \ 0] = E[ Z Z. + Z Z.Z.] . 

i=l 1 i=l 1 1=1 1 J 

Utilizing Wald’s Fundamental Identity, this may be written 
E [Z 2 | 0] = E [N] E [Z ± 2 | 0] + E[N(N-1)]E 2 [Z 1 | 0] 



where, as before, Z^ and Z. are independent and identically 
distributed. Expanding and collecting terms 



E [Z 2 1 0] = E[N](E[Z i 2 | 0]-E 2 [Z i | 0]) + E[N 2 ]E 2 [Z 1 |0] 



E [ Z 2 | 0] = E[N] V[Z i | 0] + E[N 2 ] E 2 [Z i | 0] 

Then 

P E [Z 2 | 0] - E [N] V[Z. | 0] 

E [N^] = = i , (5) 

E 2 [Z | 0] 
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with 



E[Z 2 | 0] = [l-L( 6 )]Ln 2 A + L(e)Ln 2 B 

Using Eqs . (3) and (5) in (4), an approximate Variance 
of the Sample Size is thus given by 

E [ Z 2 | 0] - E [N] V [Z . | 0] - E 2 [Z| 0] 

V[N] = ^ i 

EMZj 0 ] 

V [ Z I 0] - E[N] V[Z. I 0] 

V[N] = 5 . ( 6 ) 

E [Z ± | 0] 



Unfortunately, using Eq . (6) to calculate the approxi- 
mate Variance of Sample Size leads to negative values in 
many cases. This appears to be caused by the magnification 
of the approximations used in Eq . (3) when they are entered 
in Eq. (5). Whether this is in fact the true cause, the 
approximation in general is not good. Consequently, no 
numerical tabulations of the V[N]using Eq. (6) are included. 
However, emperically determined ’’exact” values of the V[N] 
are included. 
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IV. DISTRIBUTIONS INVESTIGATED 



The author investigated the SPRT for three common 
distributions: the Binomial, Normal, and Exponential. 

Each distribution was investigated for various parameter 
values. Each specific distribution was used in calcu- 
lating the Wald "approximate" results and in generating 
empirical "exact" results for the OC curve. Expected 
Sample Size, and the Variance of Sample Size. 

Explicit equations for the points on the Wald "approx- 
imate" Operating Characteristic Curve, the "approximate" 
Expected Sample Size Curve, and the "approximate" Variance 
of Sample Size Curve are given for these distributions using 
Eqs. (1), (3) 3 and (6). 

A, BINOMIAL DISTRIBUTION 

Suppose f(x; 0) = 0 X (l-0) 1 ” x the logarithm of the 
likelihood ratio is 



Z 1 = Ln 



-0 1-0 1-XT: 

‘/xrnr’ J 

- Q O -> 



Then 



E(Z.| 6] 



m, e,) 

QLn q ■ ^ (1~0) Ln 



f(o, ep 
f(o, e o ) 



E[Z i | 6] 




(1-e) Ln 




(7) 
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and 



V [Z ± I 6] = E[Z 1 2 | 0] + E 2 [Z 1 | 9 ] 



V[Z.| 0] = 0(1-6) |^Ln ^ TTr g- T 



9 ! 




( 8 ) 



In order to obtain an approximate OC function, the 
expression of Eq . (2) for this case. 



was solved for 0 and evaluated with various selected values 



Utilizing Eq . (9) with Eq . (1) points on the Wald approxi- 
mate OC Curve were obtained. 

By substituting Eq. (7) into Eq. (3) the Wald approxi- 
mate Sample Size curve was determined. An attempt was 
made to determine the approximate Variance of Sample Size, 
utilizing Eqs. (7) and (8) with the prior results of E[N], 
in Eq . ( 6 ) . 

The acceptance and rejection numbers were respectively 
computed from 



of h( 0) , 



0 




(9) 



20 



Ln B + nLn 



A n " 



Ln 



Ln 



(1 - e o ) 

(i-e 1 ) 
( 1-0! ) 
TTTe^- 



and 



Rn 



Ln A + nLn 



Ln 



- Ln 



(i-e o ) 

(1-0!) 

(l-ep 

n^7 



B. NORMAL DISTRIBUTION 

p 

If f(x; 6) is Normal (0 3 a ) then the log likelihood 
ratio is 



z i = ^2 [2(0i-e o )x 1+ (e^ 2 )], 



and 



e 2 -e 1 2 e ( e., -e ) 

E [ Z . | 6] = + % ° 

1 2 a a d 



and 



V[Z, 



6 ] = 



(6 l- 0 o r 



( 10 ) 



( 11 ) 



Substituting Eq . (10) into Eq . (3) points on the Wald 
approximate Sample Size Curve were computed. Eqs. (10) and 
(11) with E[N] computed above were used in Eq . (6) in an 
attempt to determine the approximate Variance of N. 

In order to determine points on the approximate OC 
curve, the expression of Eq . (2) for the present case was 
solved for h(0) a resulting in 
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Q 1 + 0 
1 o 



20 



h(0) 



e, - e 

1 o 



The acceptance and rejection numbers are, in this case, 
respectively given by 



0 +0 

A n = [a 2 /(6 1 -e o )]Ln B + n[ ° 2 1 ] 



and 



0 +0 

R n = [a 2 /(0 1 -0 Q )3Ln A + n X ] 



C. EXPONENTIAL DISTRIBUTION 

If f ( x ; 0) = 0 e“ 6x ; x_>0 , the log likelihood ratio 



is 



Z 



i 




( 6 l“ 9 o )X 



i 



so that 



E [Z ± | 6] 




( 0 l _e o ) 



( 12 ) 



and 



V[Z, 



6 ] = 



(6 l- e o )£ 



(13) 



Substituting Eq . (12) into Eq . (3)> points on the Wald 
approximate Sample Size curve were obtained. Eqs. (12) and (13) 
were used in Eq. (6) in an attempt to determine an approxi- 
mate Variance of N. 
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In order to determine values for an approximate OC 



curve, Eq . (2) was solved for 0, resulting in 



0 



h ( 0 ) ( 6 1 - 0 o ) 



Wey 



,/J 



(14) 



Utilizing Eqs . (1) and ( 1 4 ) points on the Wald approximate 
OC Curve were obtained. 

The acceptance and rejection numbers are respectively 
given by 



A = 
n 



9 1 

-Ln B + nLn - 5 — 

L 

(6l- 9 0 ) 



and 



R = 
n 



Ln A + nLn - 5 — 



( 6 l _ 0 o ) 
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V. PROCEDURE 



Computer programs, coded in FORTRAN IV, were written 
which gives points on the Wald approximate OC Curves, and 
the approximate Expected Sample Size curves. Programs 
for the Binomial, Normal, and Exponential distributions, 
each entitled ’’Approximate (Distribution)” are listed under 
Computer Programs. 

In order to evaluate the Wald approximations, companion 
programs called "exact" programs were written. These 
programs produce Monte Carlo simulation of the SPRT pro- 
cedure for the distribution considered. Each program 
determines ’’exact” points on the OC curve. Expected Sample 
Size curve and Variance of Sample Size curve. 

The simulation models sequential sampling from a 
specific known distribution with parameter values being 
inputs to the simulation. As each sample was observed, a 
corresponding acceptance and rejection number was computed 
and the test statistic was computed. These values were 
compared and the appropriate alternative was selected. 

A. DETERMINATION OF THE NUMBER OF REPLICATIONS 

In order for the simulation estimates of the Operating 
Characteristic points, L(8), (a Bernoulli parameter) to be 
useful, it is necessary to use a large number of replica- 
tions (that is, simulate the performance of many tests) so 
that with a high probability L(0) is ’’close” to the true 
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